Information-theoretic resolution of perceptual WSS 
watermarking of non i.i.d. Gaussian signals 
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ABSTRACT 

The theoretical foundations of data hiding have been re- 
vealed by formulating the problem as message communica- 
tion over a noisy channel. We revisit the problem in light 
of a more general characterization of the watermark chan- 
nel and of weighted distortion measures. Considering spread 
spectrum based information hiding, we release the usual as- 
sumption of an i.i.d. cover signal. The game-theoretic reso- 
lution of the problem reveals a generalized characterization 
of optimum attacks. The paper then derives closed-form ex- 
pressions for the different parameters exhibiting a practical 
embedding and extraction technique. 

1 INTRODUCTION 

Information hiding refers to nearly invisible embedding 
of a message within a host signal. This paper focuses 
on the data hiding problem, assuming a blind and sym- 
metric system. In the spirit of a communication problem 
one seeks the maximum rate of reliable transmission over 
any hiding and attack strategies. This rate is called the 
hiding capacity and depends on admissible distortion 
levels and on the watermark channel characterization. 

Watermarking is often regarded as a form of spread 
spectrum communication with various forms of chan- 
nel characterizations. The perceptual sensitivity of 
the host signal is often taken into account for choos- 
ing embedding sites and strength [TJ [7]. The attacks 
are often modelled as the addition of White Gaussian 
noise ( AWGN) [8l |4] , or as linear filtering plus additive 
noise O [3] . The authors in [10] show that the optimum 
attack is obtained by Wiener filtering and that to be 
maximally robust, the watermark should have a power 
spectrum matching the one of the original signal. 

The problem of robust embedding and extraction 
based on spread spectrum is revisited here in light of 
a more general model of the cover signal and of the wa- 
termark channel. Most of the approaches introduced 
so far consider that the cover signal can be modelled 
as an ergodic wide sense stationary Gaussian random 
process. This assumption is rarely satisfied for real sig- 
nals. We assume instead that it can be modelled as the 
realization of a set of independent non identically dis- 
tributed Gaussian random variables (referred to as non 



i.i.d. signals). The attack channel is considered to be of 
the type amplitude scaling and additive white Gaussian 
noise (SAWGN) [2]. The game-theoretic resolution of 
the problem with weighted distortion measures leads to 
a characterization of optimum attack domains. By max- 
imizing the watermarking channel signal to noise ratio, 
we then derive a closed-form expression of the water- 
mark spectral density corresponding to the best defense. 
The performance limits of the approach in terms of hid- 
ing capacity are then analyzed. The approach can be 
seen as a generalization of previous work to the case of 
non i.i.d. Gaussian sources, considering weighted dis- 
tortion measures and a more general SAWGN attack 
channel, with the exhibition of closed-form expressions 
for a practical embedding and extraction scheme. 

2 PROBLEM STATEMENT 

Let b = {bi,b2, . . . ,bn} with hi e Vi e 

{l,2,...,n} be the message to be embedded in a 
host signal x- Many approaches introduced so far as- 
sume that the signal x can be modelled as an ergodic 
zero-mean wide sense stationary Gaussian random pro- 
cess [H [9] . This assumption is rarely satisfied for real 
signals or for content adaptive watermarks. We as- 
sume instead that the host signal x can be modelled 
as the realization of a set of non stationary Gaus- 
sian random variables X — {Xi, X2, ■ ■ ■ , Xm} where 
Xi ~ Af{0, cTXi)- The information is then used as a key 
for indexing pseudo-random noise sequences which are 
additively combined with the signal. Let G be a n x m 
matrix composed of n pseudo-random generated vectors 
Gj E { — The watermarked signal is obtained 
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where Xi represents the i*'' site of the host signal and yi 
the corresponding watermarked site. In order to extract 
each embedded bit 6^, a correlation product between the 
vector Gj and y is computed. The term is a weight- 
ing factor allowing to adjust the amplitude (or energy) 
of the mark. In the following we derive a closed-form 
expression of this parameter in the case of SAWGN at- 



tacks, with weighted distortion measures and non i.i.d 
Gaussian cover signals. The attack channel is often as- 
sumed to be AWGN [B]. This model assumes that the 
distortion induced by the attack is independent of the 
watermarked signal, hence can hardly apply to attacks 
such as filtering and compression. More accurate mod- 
els assuming that the distortion depends on the water- 
marked signal and based on linear filtering plus additive 
noise have been considered in [51 [3]. Here, we consider 
that the attacked signal y' can be expressed as 

n 

y'i = IrVi + Si = liXi + liai ^ Gi^jhj + Si, (2) 

i=i 

where % is an attenuation factor on each watermarked 
site. This amounts to consider the attack channel as 
a SAWGN channel (amplitude scaling by the factor 7^, 
and additive white Gaussian noise of Si ^ A/'(0, (Jg.))- 

The distortion measure is defined as a weighted sum 
of the MSE on each sample of the host signal, in order to 
reflect the perceptual quality. The embedding distortion 
is therefore given by 



where C is a constant and 



Dxy — E 



^ipfnaj, (3) 



where (pi is a perceptual factor. Similarly, the expected 
attack distortion is given by 

m 
i=l 

3 MAP WATERMARK ESTIMATION 

The maximun a posteriori (MAP) estimation of the bit 
hi is defined as 



bj = argmax {P{Bj = bj | F"" = /)} 



(5) 



The a posteriori probability V = P{Bj = bj \ Y'"^ = y') 
can be rewritten (using Bayes law) as 

P(r"" = y' I B, = b,) X Pj B, = b,) 

— . (6) 



V = 



P{Y' 



y') 



Since the received vector y' is fixed, and that no a pri- 
ori knowledge on the message b is assumed, we have 
the a posteriori probability V (x P{Y'^ = y' | Bj — bj), 
where (x denotes an obvious renormalization. Assum- 
ing that the watermarked sites are independent, it can 
be shown [S] that the quantity can be expressed as a 
product of Gaussian distributions of the form Vi ~ 
^{O^fi^x, + (xUn - 1)) + <^l), i-e. as 
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The term bj represents the optimal estimator. From 
Eqn.® and watermarking channel can be seen as a 
gaussian channel. Estimator's performance can be mea- 
sured in terms of the signal to noise ratio E\j/Nq of the 
watermarking channel. This quantity is defined as the 
ratio between the energy of the embedded bit and the 
overall noise introduced by the cover signal (c^.), by 
the other embedded bits (a|(7t — 1)) and by the attack 
(i.e. cr^.). It is then expressed as 



E, E{b)) 1 
^0 ol 



i=l 



where 



Pi = 



7,f(a|^+«f(n-l))-|-a|/ 
4 GAME-THEORETIC RESOLUTION 



(12) 



(13) 



The optimization of the embedding and attack param- 
eters can be formulated as a game between an attacker 
and a hider. The attack searches the two vectors 7 and 
as minimizing the extractor performance (i.e. Eb/No) 
while maintaining the distortion below an acceptable 
level {Dxy' < D^y/^). This problem can be solved by a 
Lagrangian optimization: 

(t^, ^) = arg nhn I J, ^ ^ + a [D^y, - i?-^] | , 

where A > is a Lagrangian multiplier. From Eqn.(|3]) 
and (|12[) it appears that J\ is an additive functional. 
The optimization can then be made separately on each 
J\^i given by 

^a,4(7m ^ p^ + X^j (cri.(l - lif + "-7j^a- + cr|J 
by setting its derivatives with respect to the attack pa- 
rameters 7i and 0-5. 



d"ii 



+2\ip^ {n^,a^ - a% (1 - 7^)) , (14) 



and 

das. 



'2l!a^<ys. 



{^f{al^+aUn-l))+aiy 



+ 2X^'^as, (15) 



to zero on the validity domain. The resolution of the 
resulting set of equations leads to the following two ex- 
pressions for cr^ : 



a,- 



'X, 



(1-7.)] 



and 



It 



(16) 



(17) 



Equating Eqn. pB]) and (fTT]) leads to the optimum values 

ai 



and 
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7* {lu 



- it) (4. + naf) 



(18) 
(19) 



where = 



and 



The term 7^,^ 



represents the response of a Wiener filter. Since the 
attack parameters cr|* and 7* must verify cr|* > and 
it > 0, the solutions of Eqn. fTS)) and p9|) are valid only 
for 

> (20) 



(a-j — (fix, + 'T-Q^i ) + > 0, 



(21) 



where jj, = \f\ipia\. . This set of inequations defines 
three domains 2?i , and I?3 shown in Fig. [1] 

The optimum attacks can then be characterized in 
terms of the domains of validity of the attack param- 
eters 7i and . Let us first consider their limits of 
validity (7^ = and as^ =0). If 7* = (the marked 
value yi is erased), a minimum is obtained for cr^* = 0. 
A greater value for the additive noise will increase D^y' 
but will not decrease Ei,/Ni^. This attack is referred to 
as the Erase attack. If = 0, another minimum is 
given by 7* = 7ujj. This attack is a Wiener filtering. 
The last attack (defined for 7* > and for as* > 0) is 
a combination of filtering and additive Gaussian noise. 
This is called here the Intermediate attack. Table [1] 
gives the corresponding expressions of the cost function 
Jx,i{li,<^Si) denoted Jg, Jw and Jj for respectively the 
erase, Wiener and intermediate attacks. To find the op- 
timum attack, one has to find on each domain (defined 
in terms of ai and crxi), the attack that will minimize 
J\,i{lii<^5i)- From tableland constraints l|^ - (PT|) the 
minimum values of Jx,i{li^ in the domains of valid- 
ity of 7i and 175. , are given by Je and Jw on Vi and 
■D3 respectively (see [5] for details). Similarly, on 
J I < Je and J/ < Jw Thus, if the validity constraint 
it > of the Intermediate attack domain is satisfied, 
the optimum attack is given by the Intermediate attack 
(with parameters given by Eqn. pS)) and (fT9|) ). Other- 
wise, the attacker should use instead the Erase or the 
Wiener solution. 
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Figure 1: Domains defined by the validity constraints 
7* > and as* > (X ^ 0.2, x = 0.002, n = 1 bit). 
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.-l-af (n-l) 



+ \ 



2\/A(^.^-l + A(p2^ 
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Table 1: Cost function J\,i{^i, <ySi) for the 
of attack (Erase, Wiener, Intermediate). 



erent types 



Given the optimum attack, we then search the pa- 
rameters ai (strength of the watermark) that maximize 
Eb/No, under constraints of a maximum watermarked 
signal distortion {D™y'^). This leads to a Lagrangian 
approach: 

= arg max {J^ = Jx - x [D.y - D^^^^] } , (22) 

where x > is a Lagrangian multiplier. The 
cost function being the additive functional = 
J2T Jx.iili-, (JSi) - X [Dxy - D'xy"^] , the optimization 
can be carried out separately on each J^^i[ai) = 
Jx,iili,crSi) — x'l^Vi'^i- Let us consider the three at- 
tack strategies. In the case of the Erase attack, i.e. 
(az,crxj e Vi, Jx,iia.i) = Av?|cr^^ -xmpjal. The func- 
tion J~^^i[ai) is a decreasing function and the minimum 
valid value of ai is given by a* — \f\ipia\.. In I?2, 
setting the derivative 



dJ- 



dai 



= -2 



- 2xnipia, (23) 



to zero leads to 



Xnipfaf 



where /i 



^/Xifia'j^.. The derivative is negative for ai = fi and 
positive for = 0. The polynomial being monotonous 
on the interval [0; /i], one can conclude that Eqn. ([23| has 
a valid solution on [0; /i]. If the derivative is negative on 
I?2, the solution adopted is ai such that /iaf ~ {^i — 



o-i){'^x- +iT-'xf). Let naf 



iX = X' and nx = x' ■ 



Let us assume that cr^. is very close to af (n — 1). In the 
Wiener case ((71,0-5.) g V^), the cost function J^^i{ai) 
expressed in terms of u^. = naf is given by 



^x..«) = (l + AV?^x,) 



-X'^-Twr (24) 



Setting the derivative of Jx,i{^w ) "^ith respect to cr^. 



9Jv 



(l + AV^fiiJ 



'X, 



(25) 



'w. 



'X, 



to zero, given that a 



naf, leads to 



\ 



^x, ^ 1 + Avfcri, - vy^tc^x 



(26) 



VXn(pt 

This provides a closed-form of the optimum embedding 
parameter ai in terms of the host signal power spectrum 
(cr|. ) and for an SAWGN attack. The Wiener filtering 
can restore the signal, hence may lead to D^y' < D^y 
This can be avoided by filtering the watermarked sig- 
nal (after embedding). The distortion measure is then 
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Figure 2: Optimum values of on for X = 0.002, x 
0.0028 and n, - 100 , bits. 




Figure 3: Performances of the proposed scheme in pres- 
ence of a compression attack JPEG. 
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where ct^. — 



The reso- 



Dxy ~ X^i: 

lution of the problem leads to new parameters (see [5] 
for details): a* ~ ^/\ipia\,. Fig. [2] illustrates the vari- 
ations of the parameter oti in terms of a\, for both ap- 
proaches i.e., without (bold curve) and with a Wiener 
post-filtering (light curve) of the watermarked signal 
(with ipi = (1 -h crxj"^/^)- Unlike [1], it can be ob- 
served that for high values of , no watermark can be 
robustly embedded on the corresponding sites. 

5 RESULTS 

The approach has been tested on images against tech- 
niques using Ui — constant, and — c\xi\ [6], consider- 
ing embedding in the wavelet transform domain. Fig. [3] 
depicts the respective Eij/Nq performances in terms of 
the attack distortion. A message of 156 bits is embed- 
ded (i.e. n = 156) in the 512 x 512 gray scale Lena 
image (i.e. m = 262 144). The Lagragian multipliers A 
and X are set so that D^y = D'^y^ and D^yi = D^^y/^. 
The embedding parameters have been tuned in order to 
get the same perceptual distortion Dxy/m = 1 (with 
ipi = (\ -\- CTXi)"^^^) with the different techniques. The 
watermarked image has been attacked with a lossy com- 
pression JPEG, from 95% to 5% quality. The tests, us- 
ing the Stirmark benchmark have shown that the tech- 
nique is robust to all the non-geometric attacks. 

6 CONCLUSION 

This paper provides an information-theoretic analysis 
of information hiding in non i.i.d signals with percep- 
tual distortion metrics. Note that previous work, when 
considering perceptual watermaking was often led by 



intuition. Here we have derived closed-form expressions 
of the different extraction and embedding parameters, 
revealing an efficient and practical information hiding 
system. 
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